The Concave-Convex Procedure (CCCP)
نویسندگان
چکیده
We introduce the Concave-Convex procedure (CCCP) which constructs discrete time iterative dynamical systems which are guaranteed to monotonically decrease global optimization/energy functions. It can be applied to (almost) any optimization problem and many existing algorithms can be interpreted in terms of CCCP. In particular, we prove relationships to some applications of Legendre transform techniques. We then illustrate CCCP by applications to Potts models , linear assignment, EM algorithms, and Generalized Iterative Scaling (GIS). CCCP can be used both as a new way to understand existing optimization algorithms and as a procedure for generating new algorithms.
منابع مشابه
On the Convergence of the Concave-Convex Procedure
The concave-convex procedure (CCCP) is a majorization-minimization algorithm that solves d.c. (difference of convex functions) programs as a sequence of convex programs. In machine learning, CCCP is extensively used in many learning algorithms like sparse support vector machines (SVMs), transductive SVMs, sparse principal component analysis, etc. Though widely used in many applications, the con...
متن کاملA Proof of Convergence of the Concave-Convex Procedure Using Zangwill's Theory
The concave-convex procedure (CCCP) is an iterative algorithm that solves d.c. (difference of convex functions) programs as a sequence of convex programs. In machine learning, CCCP is extensively used in many learning algorithms, including sparse support vector machines (SVMs), transductive SVMs, and sparse principal component analysis. Though CCCP is widely used in many applications, its conve...
متن کاملOn Convergence Rate of Concave-Convex Procedure
Concave-Convex Procedure (CCCP) has been widely used to solve nonconvex d.c.(difference of convex function) programs occur in learning problems, such as sparse support vector machine (SVM), transductive SVM, sparse principal componenent analysis (PCA), etc. Although the global convergence behavior of CCCP has been well studied, the convergence rate of CCCP is still an open problem. Most of d.c....
متن کاملStructured Output Learning with Candidate Labels for Local Parts
This paper introduces a special setting of weakly supervised structured output learning, where the training data is a set of structured instances and supervision involves candidate labels for some local parts of the structure. We show that the learning problem with this weak supervision setting can be efficiently handled and then propose a large margin formulation. To solve the non-convex optim...
متن کاملCCCP Algorithms to Minimize the Bethe free energy of 3-SAT Problem
The k-sat problem is a prototypical constraint satisfaction problem. There are many algorithms to study k-sat problem, BP algorithm is famous one of them. But BP algorithm does not converge when α(constraint density)is bigger than some threshold value. In this paper we use CCCP (Concave Convex Procedure) algorithm to study 3-sat problem and we get better results than BP algorithm that CCCP algo...
متن کامل